-
Notifications
You must be signed in to change notification settings - Fork 122
introduce zig cc cross-compilation setup for GitHub Actions and local development
#2535
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
Skipping CI for Draft Pull Request. |
WalkthroughThis PR introduces Zig cross-compilation toolchain support to the build system. It adds architecture normalization and Zig installation targets to the Makefile, creates a new zigcc compiler wrapper in Go, updates the sandbox script to configure cross-compilation targeting s390x-linux-gnu.2.34, adjusts Docker runtime configurations, and expands the Python 3.12 image build matrix. Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes
Pre-merge checks and finishing touches❌ Failed checks (2 warnings, 1 inconclusive)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
… cross-compilation and unset them afterward.
…bles `CC` and `CXX`.
…, tests, and Go module setup.
…ting, and ensure `zigcc` wrappers are built
|
PR needs rebase. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 3
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
Makefile(1 hunks)runtimes/minimal/ubi9-python-3.12/Dockerfile.cpu(2 hunks)scripts/sandbox.py(2 hunks)scripts/zigcc/Makefile(1 hunks)scripts/zigcc/go.mod(1 hunks)scripts/zigcc/zigcc.go(1 hunks)scripts/zigcc/zigcc_test.go(1 hunks)wrapper.py(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
wrapper.py (1)
scripts/sandbox.py (1)
main(27-80)
🪛 checkmake (0.2.2)
scripts/zigcc/Makefile
[warning] 1-1: Missing required phony target "all"
(minphony)
🪛 Ruff (0.14.4)
scripts/sandbox.py
51-51: Local variable target is assigned to but never used
Remove assignment to unused variable target
(F841)
62-62: f-string without any placeholders
Remove extraneous f prefix
(F541)
64-64: f-string without any placeholders
Remove extraneous f prefix
(F541)
65-65: f-string without any placeholders
Remove extraneous f prefix
(F541)
84-84: String contains ambiguous × (MULTIPLICATION SIGN). Did you mean x (LATIN SMALL LETTER X)?
(RUF001)
146-146: subprocess call: check for execution of untrusted input
(S603)
148-148: subprocess call: check for execution of untrusted input
(S603)
150-150: subprocess call: check for execution of untrusted input
(S603)
wrapper.py
1-1: Shebang is present but file is not executable
(EXE001)
16-16: Consider iterable unpacking instead of concatenation
Replace with iterable unpacking
(RUF005)
18-18: Consider iterable unpacking instead of concatenation
Replace with iterable unpacking
(RUF005)
20-20: Avoid specifying long messages outside the exception class
(TRY003)
22-22: Starting a process without a shell
(S606)
| if [ "$ARCH" = "s390x" ] || [ "$ARCH" = "ppc64le" ]; then \ | ||
| PACKAGES="$PACKAGES gcc g++ make openssl-devel autoconf automake libtool cmake"; \ | ||
| PACKAGES="$PACKAGES gcc g++ ninja-build openssl-devel autoconf automake libtool cmake"; \ | ||
| fi && \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Restore make for non-ninja builds on s390x/ppc64le
Dropping make here leaves those arches without the classic build tool that many Python/C extensions still expect, so pip install of projects that call make will now fail. Please keep make while adding ninja-build.
- PACKAGES="$PACKAGES gcc g++ ninja-build openssl-devel autoconf automake libtool cmake"; \
+ PACKAGES="$PACKAGES gcc g++ make ninja-build openssl-devel autoconf automake libtool cmake"; \🤖 Prompt for AI Agents
In runtimes/minimal/ubi9-python-3.12/Dockerfile.cpu around lines 35 to 37, the
s390x/ppc64le PACKAGES list removed make which breaks builds that invoke make;
update the PACKAGES assignment to include make alongside ninja-build (e.g., add
"make" to the quoted package list) so both classic make and ninja are available
for those architectures.
| module zigcc | ||
|
|
||
| go 1.24 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Target a released Go toolchain
go 1.24 is not available yet, so current builders (Go ≤1.22/1.23) will abort with “requires go >=1.24”. Drop to the latest released version you actually need (e.g. 1.22).
-go 1.24
+go 1.22📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| module zigcc | |
| go 1.24 | |
| module zigcc | |
| go 1.22 |
🤖 Prompt for AI Agents
In scripts/zigcc/go.mod around lines 1 to 3, the module specifies "go 1.24"
which is not yet released and will abort builds on current Go toolchains; change
the go directive to an actual released version your project needs (for example
"go 1.22" or the minimum released version that supports the features you use),
update any CI/docker builders to match that version if necessary, and run go mod
tidy/build locally to verify compatibility.
| for arg in sys.argv[1:]: | ||
| if arg.startswith("-Wp,-D"): | ||
| args.append(arg.replace("-Wp,-D", "-D", 1)) | ||
| else: | ||
| args.append(arg) | ||
|
|
||
| if arg0 == "zig-cc": | ||
| args = ["/mnt/zig", "cc", "-target", "s390x-linux-gnu.2.34"] + args | ||
| elif arg0 == "zig-c++": | ||
| args = ["/mnt/zig", "c++", "-target", "s390x-linux-gnu.2.34"] + args |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fix -Wp flag rewriting
Replacing -Wp,-DNDEBUG,-D_FORTIFY_SOURCE=2 with a single -DNDEBUG,-D_FORTIFY_SOURCE=2 string prevents the compiler from seeing either define. Split the flag like the Go shim does.
- for arg in sys.argv[1:]:
- if arg.startswith("-Wp,-D"):
- args.append(arg.replace("-Wp,-D", "-D", 1))
- else:
- args.append(arg)
+ for arg in sys.argv[1:]:
+ if arg.startswith("-Wp,"):
+ args.extend(part for part in arg.split(",")[1:] if part)
+ else:
+ args.append(arg)🧰 Tools
🪛 Ruff (0.14.4)
16-16: Consider iterable unpacking instead of concatenation
Replace with iterable unpacking
(RUF005)
18-18: Consider iterable unpacking instead of concatenation
Replace with iterable unpacking
(RUF005)
|
@coderabbitai provide commentary and improvement suggestions for the |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 16240 @jiridanek: I'll provide a comprehensive analysis of the Zig cross-compilation approach with improvement suggestions. 🎯 Overall CommentaryThe Zig cc approach is innovative and pragmatic for cross-compiling to s390x from x86_64/arm64 hosts. Zig's ability to act as a drop-in C/C++ compiler with built-in cross-compilation is well-suited for this use case. However, the implementation has several areas that need refinement. 🔍 Current Architecture AnalysisStrengths:
Weaknesses:
📋 Specific Improvement Suggestions1. Choose One Wrapper ImplementationYou have both Reasons:
Action: Remove 2. Fix Incomplete -Wp Handling in Python WrapperThe Python wrapper only handles if arg.startswith("-Wp,-D"):
args.append(arg.replace("-Wp,-D", "-D", 1))But the Go version correctly handles any if strings.HasPrefix(arg, "-Wp,") {
newArgs = append(newArgs, strings.Split(arg, ",")[1:]...)
}Example failure case: 3. Make Paths and Targets ConfigurableCurrently hardcoded everywhere:
Suggestions: // zigcc.go - use environment variables with defaults
zigPath := getEnvOrDefault("ZIG_PATH", "/mnt/zig")
target := getEnvOrDefault("ZIG_TARGET", "s390x-linux-gnu.2.34")# sandbox.py - parameterize
def buildinputs(
dockerfile: pathlib.Path | str,
platform: str,
zig_version: str = "0.15.1",
zig_target: str = "s390x-linux-gnu.2.34"
) -> list[pathlib.Path]:4. Fix Makefile Architecture DetectionThe Makefile references # Add this near the top of Makefile
ARCH_RAW := $(shell uname -m)
ARCH := $(shell echo $(ARCH_RAW) | sed -e 's/aarch64/arm64/' -e 's/x86_64/amd64/')Also, the Zig download uses 5. Improve Makefile TargetThe bin/zig-$(ZIG_VERSION):
@echo "Installing Zig $(ZIG_VERSION)..."
TMPDIR=$(shell mktemp -d) # ❌ Wrong - shell executes at parse time
wget --progress=dot:giga ...Fixed version: bin/zig-$(ZIG_VERSION):
@echo "Installing Zig $(ZIG_VERSION)..."
$(eval TMPDIR := $(shell mktemp -d))
wget --progress=dot:giga https://ziglang.org/download/$(ZIG_VERSION)/zig-$(ARCH)-linux-$(ZIG_VERSION).tar.xz
tar -xJf zig-$(ARCH)-linux-$(ZIG_VERSION).tar.xz -C $(TMPDIR) --strip-components=1
rm -rf zig-$(ARCH)-linux-$(ZIG_VERSION).tar.xz
mkdir -p bin
mv $(TMPDIR) bin/zig-$(ZIG_VERSION)
chmod +x bin/zig-$(ZIG_VERSION)/zig
@echo "Zig installed successfully"6. Remove Shell Wrapper Creation from MakefileThe Makefile creates shell wrappers that are immediately replaced by the Go binary: # These are overwritten by sandbox.py later
printf '%s\n' '#!/bin/sh' 'exec /mnt/zig cc ...' > $$TMPDIR/zig-ccRecommendation: Remove these and let 7. Add Proper Error HandlingIn zigcc.go: func main() {
// Add validation
if _, err := os.Stat("/mnt/zig"); os.IsNotExist(err) {
fmt.Fprintf(os.Stderr, "Error: Zig binary not found at /mnt/zig\n")
fmt.Fprintf(os.Stderr, "Ensure Zig is mounted correctly\n")
os.Exit(1)
}
// Log what we're doing (when verbose flag set)
if os.Getenv("ZIG_WRAPPER_VERBOSE") != "" {
fmt.Fprintf(os.Stderr, "Executing: %v\n", newArgs)
}
// ... rest of code
}8. Improve Test CoverageCurrent test only covers func TestProcessArg0(t *testing.T) {
tests := []struct {
arg0 string
expected string
wantErr bool
}{
{"zig-cc", "cc", false},
{"zig-c++", "c++", false},
{"invalid", "", true},
}
// ...
}
func TestFullExecution(t *testing.T) {
// Test with mock Zig binary
// Verify argument order and transformation
}9. Add DocumentationCreate # Zig CC Wrapper
This wrapper enables cross-compilation to s390x using Zig as a drop-in replacement for gcc/g++.
## How It Works
1. The wrapper is invoked as `zig-cc` or `zig-c++`
2. It transforms arguments (especially -Wp flags) for Zig compatibility
3. It executes Zig with the target architecture set to s390x
## Known Limitations
- Zig doesn't support `-Wp,` preprocessor flags, so we split them
- Target is currently hardcoded to `s390x-linux-gnu.2.34`
## Building
## Testing
10. Handle Commented-Out Code in sandbox.pyRemove or document the commented-out CC/CXX environment settings: # f"--env=CC=/mnt/zig cc -target {target}", # Why commented out?
# f"--env=CXX=/mnt/zig c++ -target {target}",Recommendation: Either remove them or add a comment explaining the decision: # CC/CXX are unset instead of pointing to wrappers because:
# - Some build systems detect and cache the compiler
# - Letting them auto-detect ensures wrapper is found via PATH11. Remove Debug Logs from Production CodeThe large triple-quoted strings in """
Downloading jedi
× Failed to build `pyzmq==27.1.0`
...
"""These are useful for debugging but don't belong in production code. Move to a separate debug log file or remove entirely. 12. Consider Zig VersionYou're using Zig 0.15.1. Check if there's a newer stable version that might have better compatibility or bug fixes: 13. Add Dockerfile ImprovementThe RUN ls /mnt && trueRecommendation: Remove this or add a comment explaining why it's needed. If it's verifying mount availability, make it more explicit: # Verify /mnt is available for Zig toolchain mounting
RUN test -d /mnt || (echo "Error: /mnt directory not available" && exit 1)🎯 Priority RecommendationsHigh Priority:
Medium Priority: Low Priority: 🤔 Architectural Questions
Would you like me to elaborate on any of these suggestions or help implement specific improvements? 🏁 Script executed: Length of output: 42 🏁 Script executed: Length of output: 42 🏁 Script executed: Length of output: 30 🏁 Script executed: Length of output: 1148 🏁 Script executed: Length of output: 518 ⚡ Additional Finding: Zig Version Update AvailableZig 0.15.2 is available (you're currently using 0.15.1). While I couldn't extract specific bug fixes from the release notes, it's generally recommended to use the latest patch version for bug fixes and stability improvements. Recommendation: Update to Zig 0.15.2: # Makefile
ZIG_VERSION := 0.15.2All required architectures (x86_64-linux, aarch64-linux, s390x-linux) are available in 0.15.2. 📝 SummaryYour Zig cross-compilation approach is solid conceptually but needs refinement: Quick wins:
For production readiness:
The approach of using Zig for cross-compilation is excellent for avoiding complex toolchain setup. Once polished, this will be a clean solution for building s390x containers on x86_64/arm64 hosts. Would you like me to help implement any of these improvements, or would you prefer to tackle specific areas first? |
zig cc cross-compilation setup for GitHub Actions and local development
Description
How Has This Been Tested?
Self checklist (all need to be checked):
make test(gmakeon macOS) before asking for reviewDockerfile.konfluxfiles should be done inodh/notebooksand automatically synced torhds/notebooks. For Konflux-specific changes, modifyDockerfile.konfluxfiles directly inrhds/notebooksas these require special attention in the downstream repository and flow to the upcoming RHOAI release.Merge criteria:
Summary by CodeRabbit
New Features
Chores